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(57) Abstract 



The present invention relates to a method and a device in animal breeding for recording information on individual animals and/or 
collectives of animals which are within a denned area of residence, one or more video cameras being used to record video signals including 
image information from the animal or animals, and monitoring, position and/or positioning data being extracted from the recorded image 
information, comprising: analysing the recorded image information in respect of this pixel related content of one or more colour tones being 
characteristic of one or more animals and/or one or more defined portions and/or states thereof; examining, in said analysis, the colour 
content of said pixels in respect of its pertinent to one or more predefined colour tone volumes; and calculating said data on the basts of 
pixels whose colour content falls within said one or more predefined colour tone volumes. 
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ANALYSIS OF COLOUR TONE IN IMAGES FOR USE IN ANIMAL BREEDING 



TECHNICAL FIELD 

The present invention relates to a method and a 
device in animal breeding for recording information on 
individual animals and/or collectives of animals which 
are within a defined area of residence, one or more video 
cameras being used to record video signals containing 
image information from the animal or animals, and monito- 
ring, position and/or positioning data being extracted 
from the recorded image information. The invention also 
relates to use of the method and the device according to 
the invention in connection with the supervising, fodder- 
ing and milking of cows. Especially, the invention rela- 
tes to a method and a device at a milking station for 
extracting position information on the teats of an animal 
for guiding an animal-related device. j 

BACKGROUND ART 

In animal breeding, there is a general ambition to 
use technology for automation of various operations, such 
as foddering, milking, for the purpose of reducing the 
staff requirement and, consequently, keeping the costs 
down. At the same time, such technical equipment is re- 
quired that the qualities in animal breeding can be main- 
tained or improved. The technology used should also be 
possible to manufacture and service at costs that would 
make it applicable not only in large-scale breeding, but 
also in small and medium-size animal breeding plants. 

A common requirement is that it should be possible 
to identify and/or determine the position of individual 
animals, collectives of animals, as well as defined parts 
of animals, e.g. to be able, when foddering, to portion 
the correct amount of fodder, correctly composed, or to 
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be able to establish the position of the udder and teats 
of a cow at a milking station. 

The problem in identifying cows in the first place 
has been solved by providing each individual with a so- 
called transponder which contains ID information on the 
cow and can be read as the cow passes defined interro- 
gation stations. However, in a more general monitoring 
of stock of animals, there remains a wish to be able to 
determine the position as well as, where appropriate, 
identify both individual animals and collectives of ani- 
mals without necessitating the provision of said trans- 
ponders and the cows passing an interrogation station. 
For instance, this may involve the monitoring of animals 
moving freely in an enclosed pasture or cow-house. 

In many applications, great accuracy is required 
at the same time as the sensing should be able to take 
place at a great or small distance. This has resulted 
in methods and systems that are based on image informa- 
tion from the animals, the animal or portions of the 
animal, which is recorded by one or more video cameras. 
These cameras can be of conventional type and the image 
information can be recorded in black and white, but pre- 
ferably the recording takes place in colour to permit the 
most accurate evaluation. The video cameras can be of the 
type operating with an electronic tube, but are prefer- 
ably of the CCD type, having, for instance, the advantage 
of a smaller weight and a construction which, generally 
seen, is more durable. By using standard-type video came- 
ras, the costs can be kept down, at the same time as it 
is possible to work with a well-developed technology. 

Regarding milking and foddering, there are several 
motives for automated milking. Thus, one wants to reduce 
the staff requirement in order to keep the costs down and 
at the same time eliminate the ergonomically unsuitable 
operations in connection with milking. An automated 
milking procedure also makes it possible to increase the 
number of milking occasions a day, and to perform milking 
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independently of the time of the day, which has been 
found to result in a considerably increased milk produc- 
tion. 

Milking stations of the type in question are intend- 
5 ed for cattle moving freely and finding the milking sta- 
tion on their own. During milking, the cow is standing in 
a stall which is closed by an automatically operated gate 
or like means, which makes the cow take a reasonably well 
-defined position in the stall. In order to eliminate the 

10 risk of injuries, one strives to provide a quiet milking 
situation, which makes the cow stay fairly calm. This 
requires in turn that the milking procedure is carried 
out quickly and with great accuracy. Even under favour- 
able conditions, one cannot expect the cow to stay quite 

15 calm in the stall, since there is a certain space for 

movements. Moreover, the individuals behave in different 
manners. This results in the udder and teats of the cow 
moving, which as a rule causes problems in the automatic 
application of teat cups and/or the guiding of other 

20 equipment in the context. 

Generally seen, there are two main techniques of 
applying the teat cups: on the one hand at the same time 
as a milking means, where the teat cups are located in a 
holder with adjustable teat cup positions by a robot arm 

25 being raised towards the udder and, on the other hand, by 
applying the teat cups one by one to the respective teat 
by means of a guidable robot arm which collects the teat 
cups from a rack. Preferably the latter method is used 
since it allows greater flexibility and accuracy. 

30 Prior art presents many attempts at solving the 

problem in the automatic application of teat cups. For 
instance, use has been made of sensors, e.g. IR sensors 
which sense, at a distance, the position of udder and 
teats in the horizontal plane, supplemented with addi- 

35 tional sensors of the same or a corresponding kind, 
arranged on or adjacent each teat cup for accurately 
adjusting its position when the teat cup is raised 
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towards the udder/teat. The experiments with this type 
of arrangement and the like have, however, not been 
satisfactory so far. 

The development in this field has also led up to the 
suggestion of using image information, preferably colour 
image information, from one or more video cameras and 
subsequent image analysis for controlling the application 
of teat cups. The information content in a colour image 
is very great and thus provides the conditions for 
achieving the desired accuracy. The great amount of 
information is, however, in conflict with the demand for 
high speed information processing and accompanying excel- 
lent dynamics and precision in the system. Rapid process- 
ing of large data sets requires a great computer capa- 
city, which in turn causes problems in manufacturing a 
technical installation at costs that can be defrayed by 
the expected savings. The computer power costs also 
render it difficult to develop a milking station which 
is suited not only for large-scale production but also 
for medium-size and small stock of animals. 

An example of prior-art technique using video came- 
ras for recording image information is to be found in GB- 
A-2,272,971, which discloses the use of two video cameras 
for recording colour image information at a milking sta- 
tion for the purpose of ascertaining whether the udder 
and teats of the cow are dirty or injured. This publi- 
cation contains no detailed description of the process- 
ing of the image information, but it is established that 
colour images can be used and processed in real time, 
provided that a sufficient data processing capacity is 
available. 

DE-37 44 867 C2 also discloses the use of video 
cameras at a milking station for applying teat cups to 
the cow's udder. The locating of a teat is intended to 
take place by so-called contour comparison between a 
significant contour characteristic of a teat, such as the 
tip thereof, in a recorded image and a reference contour. 
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Nothing is said in this patent specification about the 
manner in which this significant contour information 
should be extracted from a recorded video image and how 
this significant contour information is then compared 
5 with a reference contour. This type of image or contour 
comparison usually implies large sets of data, thus 
necessitating a great data processing capacity. The use 
of the method at a milking station also requires that the 
-image processing be carried out in real time, which 

10 further increases the demands for available data 
processing capacity. 

The use of colour image information to achieve the 
greatest possible information accuracy consequently is 
in direct conflict with the demand that the technical 

15 installation involved should be possible to manufacture 
at costs that are acceptable in the situations of use. 
It is possible to get an idea of the data quanti- 
ties involved by establishing that an image frame of the 
video information consists of two image fields, and that 

20 « the field frequency is e.g. 50 Hz. From a current type 
of video camera, there is obtained for each field a 
data quantity consisting of 768*290 pixels (the number 
of pixels per line * the number of lines per field) , 
where each pixel in turn contains 24 bits of colour 

25 information. If it is required that the processing of 
this data quantity occur in real time, i.e. within the 
recording time of the field, the available space of time 
is 20 ms, which corresponds to a processing speed of 
about 2 67 Mbit/s. Processors that are available today do 

30 not cope with this processing speed, especially if they 
are processors in so-called built-in systems, which are 
also required to manage input/output, have a moderate 
consumption of power and involve purchase costs that are 
reasonable in the context . 
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OBJECT OF THE INVENTION 

An object of the invention is to provide an improved 
method device for processing animal related image data. 

SUMMARY OF THE INVENTION 

According to the invention, there is provided a 
method and a device in animal breeding for recording 
information on individual animals and/or collectives of 
animals which are within a defined area of residence, one 
or more video cameras being used to record video signals 
including image information from the animal or animals, 
and monitoring, position and/or positioning data being' 
extracted from the recorded image information, compri- 
sing: analysing the recorded image information in respect 
of this pixel related content of one or more colour tones 
being characteristic of one or more animals and/or one or 
more defined portions and/or states thereof; examining, 
in said analysis, the colour content of said pixels in 
respect of its pertinent' to one or more predefined colour 
tone volumes; and calculating said data on the basis of 
pixels whose colour content falls within said one or more 
predefined colour tone volumes. 

It has been found that this concept mades it 
possible to significantly reduce the amount of time and 
memory capacity required for processing image data when 
recording information on animals and animal related fea- 
tures. Especially, this reduction of processing required 
enables real time processing of image information, there- 
by making it possible to control other animal related 
processes in real time as well. Furthermore, this pro- 
cessing can be accomplished by simpler and less expensive 
processors. 

According to one advantageous aspect of the inven- 
tion, there is provided a method and a device in animal 
breeding for recording information on individual animals 
and/or collectives of animals which are within a defined 
area of residence, one or more video cameras being used 
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to record video signals containing image information from 
the animal or animals, and monitoring and/or position 
data being extracted from the recorded image information, 
wherein: the RGB signals included in the video signals 
5 are produced in digital form for the pixels in a frame or 
field of the video information; the digital RGB signals 
for the pixels are analysed in respect of their content 
of one or more colour tones which are characteristic of 
one or more animals and/or one or more defined portions 

10 and/or states thereof; each characteristic colour tone is 
identified by applying a pertainment function, the colour 
content of a pixel being tested for its pertainment to 
one or more colour tone volumes defined in advance and 
constituting one or more parts of a colour tone space 

15 formed by the colour intensities of the RGB signals; 

pixels are selected, whose colour content falls within a 
colour tone volume; and said data are calculated on the 
basis of the thus selected pixels. 

The conversion of the RGB signals by using said per- 

20 ' tainment function in fact results in a data reduction by 
the factor 3. In dependence on how well the searched 
characteristic colour tones are defined by said one or 
more colour tone volumes,- a totally quite considerable . 
data reduction can be achieved. When the RGB signals are 

25 analysed with respect to a characteristic colour tone 

represented by a well-defined colour tone volume, which 
may be the case when, for instance, one wants to decide 
whether one or more animals are staying within the defin- 
ed area or not, implying that the characteristic colour 

30 tone selected for the animal /animals is to be distin- 
guished from the colour tones of the surroundings, this 
processing of the image information can be sufficient to 
create the desired monitoring or positioning information. 
For the description of the invention in the speci- 

35 fication and claims, the RGB signals have been selected 

as the form of representation for the colour tone content 
of the image information, which can be said to imply that 
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the colour information is related to a three-dimensional 
RGB colour tone space. However, it will be appreciated 
by those skilled in the art that it would just as well be 
possible for the colour content of the image information 
to be represented by corresponding luminance and chromi- 
nance signals, implying that the colour information is 
accounted for in polar form, and that measures correspon- 
ding to those defined in the claims can be defined for 
this mode of representation. 

Preferably, every pixel within a colour tone volume 
is given a corresponding pertainment value which is 
indicative of the position of the pixel in the colour 
tone volume and, thus, is a measure of the relationship 
degree of the pixel. 

By using these pertainment values, a further dis- 
crimination in the amount of data can be carried out, 
which according to the invention can take place by, for 
instance, setting a threshold, implying that merely per- 
tainment values which are greater than a given threshold 
value are selected for the continued processing. An 
alternative or supplementary possibility of continued 
data reduction is averaging between pertainment values 
representing adjoining pixels along a line in the image 
information and excluding average values below a prede- 
25 termined threshold value. One more possibility is to 

derive the pertainment values for the purpose of estab- 
lishing changes between the pertainment values of adjoin- 
ing pixels. By then maintaining merely derivative values 
above a certain size and discriminating the others, it is 
possible to isolate pixels lying in the edge zone between 
areas having conceivable differences in colour tone. For 
instance, this makes it possible to detect the contour of 
an object having a characteristic colour tone against a 
background. For example, it is possible to select pixels 
relating to probable teat edges or animal body edges and 
therefore having a great steepness/amplitude difference 
between adjoining pertainment values. A further proces- 



20 



30 



35 



WO 97/15901 




9 



PCT7SE96/01368 



sing possibility is the forming of average values of 
adjoining pertainment values and then discriminating 
average values below a threshold value. 

By these supplementary measures, a drastic reduc- 
5 tion of the amount, of data is achieved, which renders 
it possible to obtain the desired information from the 
remaining data at a high processing speed while using 
a restricted processor capacity, for instance obtaining 
- coordinates for portions of the observed animal. 

10 A further increase of the final processing speed 

can be achieved by passing, for the reduced data set for 
relevant pixels, from so-called bitmap representation 
implying that pixels are processed at a constant frequen- 
cy, to coordinate representation implying that merely for 

15 the relevant pixels,, the associated x coordinates are 
processed along a image line, which requires that the 
relevant pixels be processed line by line and implies 
that the y coordinate can be considered known by the 
position of a current line in the image being known. 

20 According to a the invention, said data reducing 

means are realised in a dedicated hardware, which is 
adapted to operate with a constant pixel delay and which 
comprises controllable delay means for compensation for 
a variable consumption of time for various selectable 

25 processing procedures, while said calculating means are 
adapted to measure the used calculating time. Preferably 
every colour tone volume is defined by a look-up table of 
pertainment values that are stored in a look-up memory 
unit, to which the RGB intensity values of the pixels are 

30 supplied as address information, the pertainment value 
for each pixel being read from the memory if the colour 
tone of the pixel is related to the colour tone volume 
defined by the look-up table or, alternatively, the per- 
tainment value zero or the equivalent is given to the 

35 pixel if it falls outside said volume. 

There are several advantages of performing the 
reduction by conversion from RGB signals into pertainment 
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values in a hardware solution. Thus, it is possible to 
make a considerable saving of time in the processing 
compared with a processor-based software solution. Depen- 
ding on the design of the software solution, the estima- 
ted saving of time may amount to a factor of about 200- 
1000. If the software solution is so designed that the 
conversion need not be carried out for all pixels in the 
image information, the saving of time can be estimated at 
a factor of 100. The use of a hardware solution also 
creates conditions of operating with a deterministic, 
constant pixel delay through the image analysing system 
in its entirety. This is of importance when using the 
method according to the invention in a dynamic system, 
e.g. for establishing the position coordinates for a 
moving object. Thus, knowing the pixel delay and having 
the use of a dynamic model for the "normal" movement 
pattern of the object, it is possible to predict the 
momentary position coordinates, thereby compensating for 
the delay and obtaining a truer account of the actual 
20 f position. 

According to a preferred embodiment of the inven- 
tion, in which two spaced-apart video cameras are used 
for recording the image information, a further reduction 
of the amount of data can be achieved by illuminating the 
currently monitored object by means of two limited light 
beams of different colour, one light beam preferably 
containing light within the red colour spectrum, while 
the other light beam contains light within the blue 
colour spectrum; by directing said light beams to the 
object, such that the light beams are caused to intersect 
within an expected area of residence of the object, which 
has been predicted on the basis of historical data regar- 
ding the object; and by selecting only pixels containing 
colour contributions from the two light beams. 

These measures enable discrimination of pixels which 
are related to a current colour tone volume, but which 
refer to objects both in front of and behind the expected 
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area of residence of the object and will, as a result, 
only be illuminated by light of one of the colours and 
may consequently be discriminated for this reason. Even 
if the image information in itself is two-dimensional, 
5 discrimination in the third dimension is enabled in this 
manner, i.e. in depth relative to the direction of obser- 
vation of the video cameras. This type of discrimination 
is particularly valuable when the image analysis aims at 
producing pixels that coincide with the contour of the 
10 object in surroundings where the object can be both con- 
cealed by and appear against the background of objects 
having the same or essentially the same colour tone as 
the object. 

A device according to the invention for calculating 

15 . monitoring and position data for individual animals and/ 
or collectives of animals which are within a defined area 
of residence, comprises briefly one or more video cameras 
for recording image information from the area of resi- 
dence, signal converting means for producing the digital 

20 RGB signals of the pixels, a data reducing unit having 
a pertainment function unit giving every pixel an asso- 
ciated pertainment value according to its content of an 
animal-characteristic colour tone, memory means for stor- 
ing selected pixels, and calculating means for calculat- 

25 ing said data on the basis of the selected pixels. 

Preferably, according to the invention, use is made 
of a so-called look-up memory for accomplishing the per- 
tainment function unit. The look-up memory is programma- 
ble with a look-up table of pertainment values defining 

30 the animal-characteristic colour tones. 

It has been found possible, on the conditions that 
apply to the image information in connection with animal 
breeding, to develop colour-tone-based look-up tables 
which enable a far-reaching but yet well-balanced reduc- 

35 tion of the data set. 

According to a specific aspect of the invention, 
there are provided a method and a device at a milking 
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station for extracting position information on the teats 
of an animal for guiding an animal-related device, one or 
more video cameras being used for recording image infor- 
mation from the udder, and data for calculating the posi- 
tion information being extracted by colour analysis of 
the image information, wherein: the digital RGB signals 
of the pixels in the recorded image information are 
generated; the number of pixels are reduced by analysis 
.of the RGB signals with respect to the content of colour 
tones which are characteristic of the colour of the 
teats; the analysis comprises application of a pertain- 
ment function, which implies that the colour content of 
the pixels is tested for its pertainment to a limited 
colour tone volume, defined in advance, of teat colour 
tones in an RGB intensity space and is given a pertain- 
ment value corresponding to the positions of the pixels 
in or outside the colour tone volume; pixels having a 
colour tone that falls within the colour tone volume are 
selected as probable teat pixels, and the pertainment 
values of the selected pixels are used for continued 
processing and real time calculation of the positions of 
the teats. 

Hence, the inveniton provides a method and a device 
which has been found to enable teat cup application with 
high performance in respect of accuracy, rapidity and 
dynamics . 

One more possibility according to the invention 
of processing the pertainment values implies that one 
selects pixels in a given spaced-apart relationship and 
tests whether a teat edge transition can be imagined to 
occur between the elements, which may be carried out, 
for instance, by derivation or amplitude comparison. 'sub- 
sequently, pairs of values are discriminated, for which 
such a transition could not be established. This method 
can be particularly usable in case of major image infor . 
mation interference. 
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Preferably, the data set can be further reduced by 
illumination of the teats with light beams of different 
colours from two directions and of such orientation that 
the light beams intersect adjacent the teats. By adapta- 
5 tion of the pertainment function, it is possible to limit 
the number of pixels/pertainment values to only such as 
contain colour contributions from both light beams. As a 
result, discrimination of pixels in "depth" of the re- 
corded image is achieved, which may confer considerable 

10 advantages, for instance, when the position of a teat is 
to be established against a background of essentially the 
same colour tone, e.g. one hind leg of the cow. 

An algorithm for determining the teats suitably 
starts from the teat tips, i.e. the analysis of the 

15 recorded frame/field should begin from below. This means 
that the analysis is delayed by the recording time for 
the frame/field, which in an imaginary case means 40 and 
20 ms, respectively, since a video camera normally scans 
the image from the upper left corner to the lower right 

20 corner. Preferably, this delay can be eliminated by the 
video camera being arranged to scan the image from below 
and upwards and by continuously analysing the recorded 
image lines. This can readily be achieved by using one or 
more video cameras of standard type which are mounted 

25 upside-down. A continuous processing of the image infor- 
mation means that each field is analysed separately. The 
fields are interlaced, which means that the vertical 
resolution is halved compared with an analysis of a 
frame. Since the resolution in the vertical direction is 

30 uncritical for determining the positions of the teats, 

the deteriorated resolution can be ignored in favour of a 
saving of time of 20 ms, i.e. the reading time for a 
field. 



35 prises animal-related means, which are guidable towards 
' to the teats of an animal in dependence on position 

information for the teats, at least one video camera for 



A milking station according to the invention com- 
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recording image information from the teats and/or udder 
of the animal, and a device for extracting, by colour 
analysis of the image information, data for calculat- 
ing the position information, and includes, for proces- 
sing the video information, signal converting means, data 
reducing means, memory means, calculating means and 
control means, all having the detailed features that will 
appear from the claims. 

The use of a pertainment function for limiting the 
amount of information in the manner described above has 
been found to satisfy the demands on rapidity and accu- 
racy in the establishment of the teat position, indepen- 
dently of the significantly varying conditions at a milk- 
ing station of the type involved. For instance, the light 
conditions may thus vary to a considerable extent, at the 
same time as the variations in teat colour can be very 
great between different animal individuals. This can be 
explained by the fact that the method using a pertainment 
function renders it possible to operate with more or less 
"diffusely" limited colour volumes, which increases the 
possibilities of identifying pixels of interest under 
varying conditions. By using a well-balanced look-up 
table of pertainment values, the amount of information 
can be reduced, to a few per cent of the original amount. 

By the term "milking station" is not meant that the 
station is exclusively usable for milking, but it can 
also be equipped with other functions. Thus, the station 
may be, for instance, a combined foddering and milking 
station, and also comprise other equipment for the 
cleaning and disinfecting of teats and udders. 

Further embodiments of the method and the device 
according to the invention will be evident from the 
appended claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The invention will be described in more detail with 
reference to non-restrictive embodiments as illustrated 
in the drawings, in which: 
5 Fig. 1 shows a fundamental arrangement at a milking 

station according to a first exemplified embodiment of 
the invention. 

Fig. 2 shows parts of a technical installation at a 
milking station according to the invention, 
10 Figs 3a, b, c show alternative animal-related 

devices at a milking station according to the first 
embodiment of the invention, 

Fig. 4 is a top plan view of the fundamental arran- 
gement when illuminating teats by differently coloured 
15 light sources in a milking station according to the first 
embodiment of the invention, 

Fig. 5 is a block diagram of a image analysing 
device according to the invention, 

Fig. 6 is a block diagram of data reducing means 
2.0 included in the device according to Fig. 5, 

Fig. 7 is a flow diagram illustrating the overall 
mode of function of the device according to Fig. 5, 

Fig. 8 is a flow diagram of the routine when calcu- 
lating the pertainment values in a look-up table, and 
25 Fig. 9 shows a fundamental arrangement for recording 

monitoring information according to yet another example 
of an embodiment of the invention. 

Equivalent elements in the various Figures have been 
provided with the same reference numerals. 

30 

DESCRIPTION OF EMBODIMENTS 

For the sake. of clarity, the following description 
will be directed in the first place to the milking of 
cows and the automatic application of teat cups. However, 
35 he usability of the invention is not restricted to merely 
milking procedures, but is just as well usable for 
guiding animal-related devices of different kinds or for 
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different types of monitoring of. animals and/or parts or 
states thereof in general, as will be described later. 

With reference to Fig. 1, a cow 1 is standing in a 
milking and/or foddering station formed as a stall. The 
5 stall is formed of two equivalent lateral portions made 
of bars 2. The lateral portions are connected to a front 
stall part 3, in which foddering takes place. The rear 
part of the stall can be closed by an automatically ver- 
tically adjustable gate means 4, which in the lowered 
10 position as shown prevents the cow from moving backwards 
out of the stall. The gate means 4 is operated by a sen- 
sor (not shown) which is arranged in the stall and senses 
the cow's entering the stall. 

Moreover, Fig. 1 shows two video cameras A and B, 
15 which are arranged in the same horizontal plane in a cer- 
tain spaced-apart relationship and which are directed to 
record image information from the cow's udder and teats 
5. Schematically shown teat cups 6 can be applied to the 
teats by means of a controllable robot arm 7. 
20. The procedure is essentially as follows. When the 

cow 1 is correctly positioned in the stall, said sensor 
senses this and releases the lowering of the gate means 
4. The video cameras A and B record image information 
from the teats ,5, and information on the positions of 
25 the teats are prepared from the image information. This 
information is used for controlling the robot arm 7 and 
applying the teat cups 6. 

As shown in Fig. 1, the cow is, during milking, shut 
up/kept in place in the stall by said gate means 4 . It is 
therefore essential to carry out the application of the 
teat cups quickly and with maximum accuracy, thereby pro- 
moting a calm milking situation and reduced risks of 
injury. 

It is pointed out that the mechanical design of the 
35 station is of no import to the invention and therefore is 
not shown in detail. In Fig. 1, the video cameras A, B 
are arranged on the same side of the cow and are both 
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oriented to record image information from the teats 
obliquely from the front between the forelegs and hind 
legs of the cow. The use of two video cameras makes it 
possible to calculate all coordinates XYZ based on video 
5 information only. 

The arrangement in Fig. 2 shows a milking robot 8 
comprising the robot arm 7, which is operable horizontal- 
ly, in depth and vertically and which at the very front 
. is fitted with a gripping means 7a, and a computerised 

10 control device 9. The gripping means 7a supports a teat 
cup 6 which is to be applied to a teat 5. The robot is 
arranged to take out the teat cups 6 one by one from a 
teat cup rack (not shown) . The Figure shows the video 
camera A of the two video cameras A, B in Fig. 1. The 

15 video camera B is imagined to be concealed by the camera 
A and positioned at a certain distance beyond this. The 
cameras A, B are directed to the teats 5 for recording 
image information from these. The video cameras A, B are 
connected to the control device 9 via the transmission 

20 line 10 for transferring the recorded image information. 

The guiding of the robot arm 7 with the teat cup 6 
takes place first of all with regard to the tip 5a of 
a teat 5. Based on the recorded image information from 
the video cameras A, B, the XYZ coordinates for the teat 

25 tip are calculated, and this coordinate information is 

used for controlling the robot arm 7 . The XYZ coordinates 
are calculated by application of simple trigonometry and 
must be considered obvious. It has therefore not been 
found necessary to burden the present specification with 

30 a detailed account thereof. In this context, reference is 
instead made to the above-mentioned DE 37 44 867 C2 . 

In an alternative embodiment, at least one of the 
video cameras A, B can be arranged on a carrier, which 
in turn is supported by the robot arm 7. Such an arrange- 

35 ment creates possibilities of correcting a calculated 
teat position with regard to the dynamic properties of 
the robot arm. 
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The method according to this first embodiment of the 
invention can also be used to guide animal-related 
devices of some other kind than teat cups 6. Figs 3a, 3b, 
3c show alternative animal-related devices. Thus, Fig. 3a 
shows a video camera 11 which can be moved, by the robot 
arm 7, towards the teats 5 in order to record close-up 
images thereof, for instance, for examining an injured 
udder and/or teat, dirtying thereof etc. Fig. 3b shows a 
.spraying device 12 for the spraying of udder and teats, 
for instance, for the purpose of cleaning. Fig. 3c shows 
a cup which is filled with iodine up to a suitable level, 
in which the teats are immersed when the milking is 
finished. Additional alternatives are conceivable, for 
instance, a teat cleaning device, comprising two 
counterrotating rollers between which the teat is caught 
and cleaned. 

Fig. 4 is a schematic top plan view of an arrange- 
ment of the two video cameras A, B directed to the teats 
5 indicated by circles. The video cameras have the visual 
fields A 1 and B», respectively. The hind legs of the 
cow are represented by the elliptic forms 14 and 15. As 
appears from the Figure, the cameras A and B also catch 
image information from the inside of the posteriorly 
situated leg 15.. In the case where a corresponding por- 
25 tion of the cow's leg contains colour tones which are 
also characteristic of the teats 5 and which therefore 
are within the colour tone volume used for the data 
reduction, it may be difficult to distinguish "true" teat 
pixels from pixels relating the hind leg of the cow. This 
increases the amount of data and makes the calculation of 
the position information for the teats more difficult. 

The difficulty is overcome by using two light 
sources C and D and their associated light-focusing means 
16 for generating a directed light beam. The source C 
emits a red light beam C to the expected area of resi- 
dence of the teats 5, while the light source D emits a 
corresponding blue light beam to the same area. The light 
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sources C and D are arranged, such that the light beams 
C and D' intersect at a certain angle in the teat area. 
Posteriorly situated portions of the hind leg 15 are 
illuminated either by one or by the other light beam, but 
5 not by both light beams at the same time. Since the teats 
5 are illuminated at the same time by both light beams, 
the "teat pixels" can be distinguished from the "hind leg 
pixels" by the look-up table, which defines the relevant 
- colour tone volume, being given such a design that only 

10 pixels with colour contributions from both the red and 
the blue light beam fall within the colour tone volume, 
thereby making it possible to discriminate the remaining 
pixels. Even if the recording of images is in itself two- 
dimensional, a pixel discrimination is in this manner 

15 accomplished in the third dimension, i.e. in depth. The 
amount of information which is thus further reduced 
implies that the calculation of the position information 
will be accelerated and also will be more accurate. 



20 camera 17 which preferably is of the CCD type. In the 
block 18 (VIDEO/RGB) , the analog video signal from the 
camera 17 is converted into digital RGB signals of the 
pixels of the fields. The digital RGB signals are 
supplied to a data reducing means 19 (REDUC) , where the 

25 RGB signals of the pixels are analysed in respect of 

colour by using a pertainment function based on a look-up 
table. In the manner as will be described in more detail 
with reference to Fig. 6, the RGB signals of the pixels 
are converted into the corresponding pertainment values, 

30 and pixels satisfying the requirements of the pertainment 
function are selected. The pertainment values of the 
selected pixels are transmitted to a memory 20 (MEM) , 
either directly or only after additional data reducing 
processing. On the basis of the pixel information in the 

35 memory 20, a central processing unit 21 (CPU) calculates, 
by means of selectable calculation algorithms, the desi- 
red monitoring, position and/or positioning data, which 



The block diagram in Fig. 5 shows a video 
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for example may be teat position information that is 
supplied to the milking robot 8 or animal behaviour 
information that is supplied to a operator monitor (not 
shown), here represented by a general operation block 22 
(OP) . It should be emphasised that the blocks 17-21 
represent functions, which do not necessarily correspond 
to a corresponding division in their physical accomplish- 
ment . 

The communication of the conventional analog video 
signal between the video camera 17 and the block 18 
occurs through a coaxial cable. For the communication of 
control signals between the blocks 18 and 19, there is a 
standardised I 2 C bus, and for said digital RGB signals 
and the necessary clock and synchronising signals, there 
15 are dedicated data lines. From the data reducing block 
19, the memory block 20 receives the pertainment values 
of a reduced number of pixels in a field or, alternati- 
vely, pertainment values after additional data reduction 
by derivation or averaging of the pertainment values and 
20 subsequent discrimination. There is also the possibility 
of supplying to the memory 20 pertainment values, deriva- 
tives or average values for merely pixels of interest, 
represented in coordinate form instead of bitmap form. 

The memory, 20 has capacity for storing the pixels in 
25 a field or, alternatively, a complete frame of the video 
information consisting of two interleaved and interlaced 
fields. The memory 20 has double input and output ports 
allowing the processor in the calculation block 21 to 
read information for already stored lines in the field, 
30 at the same time as information for subsequent lines is 
received from the block 19 for input or writing. 

Monitoring, position and/or positioning data calcu- 
lated in the block 21 are used for example for controll- 
ing a milking robot 22 (8) for guiding a teat cup or some 
35 j other animal-related device according to Fig. 3. The cal- 
\ culated data may however be used for other purposes, such 
1 as for controlling an alarm transmitter in an operator- 
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manned place or a control means for controlling other ( 
kinds of animal-related devices, incorporated in the 
general operation block 22, or for monitoring animal 
behaviour in general. 
5 When using two or more video cameras, the inventive 

device comprises for each video camera the same set of 
functional blocks as in Fig. 5. 

Fig. 6 is a more detailed block diagram for said 
- data reducing means 19 in Fig. 5. These have three 

10 inputs, designated R, G and B for the respective colour 
components in the RGB signals from the block 18. The 
input signals conform with standards and consist of so- 
called 8 bits RGB in 4:4:4 format and are generated by 
standard circuits in the block 2, which are well known 

15 to those skilled in the art. The current type of circuit 
is made by, for instance, Philips. Each input is provided 
with a scaling means 23, 24 and 25, respectively (SCAL) . 
The function of the scaling means is to amplify the dyna- 
mics of the colour signals and simultaneously adapt the 

20 input signals to the address format for a subsequent 

look-up memory 26. To enable quick reprogramming of the 
look-up memory, the resolution has in the present case 
been limited to 5 bits per colour. 

The output signals from the scaling means 23, 24, 25 

25 are supplied on the one hand to the address inputs of the 
look-up memory 26 (LOOKUP) and, on the other hand, to the 
respective output multiplexors 30, 31, 32. 

The look-up memory 26 is accomplished by a rapid 
static RAM memory. The look-up memory can be quickly 

30 reprogrammed with a look-up table of pertainment values, 
which is adapted to the situation of use, for instance, 
a look-up table defining a colour tone volume of colour 
tones which are characteristic of an individual animal or 
a group of animal or of specific features thereof such as 

35 the teats of an individual animal or a group of animals 
having a similar teat colour. The preparation of the 
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pertainment values of the look-up tables will be 
described in more detail with reference to Fig. 8. 

The output signal from the look-up memory 26 is a 
series of pertainment values of 8 bits, which means that 
the supplied three RGB signals of 8 bits each have been 
converted into a signal of 8 bits. In any case, a data 
reduction to 1/3 of the original amount of data has thus 
been performed by the look-up unit. 

The output signal from the look-up unit 26 is sup- 
plied on the one hand to a calculating or counting unit 
21 (ALU) and, on the other hand, to the output multi- 
plexor 30. The calculation unit comprises means for 
deriving supplied pixel samples /pertainment values or 
performing averaging between adjoining samples. In a 
15 simplified embodiment, the deriving can be approximated 
to an establishment of the difference between adjoining 
samples. The calculation unit 27 is preferably implemen- 
ted by a programmable circuit, which is reprogrammed for 
selecting the desired function. The output signal from 
20 ( the calculation unit 27 is supplied to a discriminator 28 
(DISC) , which serves to establish whether the processed 
signal from the calculation unit 27 is within or outside 
predetermined limit values and, based on this, decide 
whether the corresponding pixel data should be read into 
25 the memory 20. The output signal of the discriminator is 
supplied to the output multiplexor 30. 

The block 2 9 (HOR.POS) represents a function for 
determining the horizontal position of a pixel of inte- 
rest along a line in the field. This position is stored 
in the memory when a selected sample/derivative/average 
value is output from the discriminator 20. Since the Y 
coordinate of the pixel is known from the position of the 
line in the field and the memory position in the memory 
20, the processor 21 gets access to the coordinates for 
35 these pixels of interest. When the pixel information is 
obtained via the discriminator, the information thus is 
transformed from bitmap form to coordinate form. In the 
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embodiment shown, the output multiplexor 31 supplies 
one part of the horizontal position and the output multi- 
plexor 32 the other part thereof. 

The output multiplexors 30, 31, 32 are used to se- 
5 lect output data for the memory 20. The following selec- 
tions are possible: colour information of all picture 
elements or pixels is let through without processing by 
the output multiplexors 30, 31, 32 supplying the inten- 
_ sity values for red, green and blue' directly from the 

10 associated scaling means 23, 24 and 25, respectively; the 
output multiplexor 30 supplies all pixels of the correct 
colour tone from the output of the look-up memory 26; the 
output multiplexor 30 supplies pertainment values after 
deriving or, alternatively, averaging for "pixels of 

15 interest", in which case the output multiplexors 31, 32 
simultaneously supply said first and said second part of 
the horizontal position of the pixels. 

For controlling the mode of operation, control means 
33 (CON) are included, which receive a synchronising/ 

20 clock signal (SY/CL) from the block 18 (Fig. 5) and gene- 
rate control signals for all the included blocks 23-32, 
which has been indicated by dotted lines. By these con- 
trol signals, the selection of said output data is con- 
trolled, as well as the selection of function in the 

25 calculation unit 27. Moreover, the programming of the 

look-up memory 26, the setting of the scaling means 23-25 
and the selection of discrimination levels in the discri- 
minator 28 are controlled. In reality, the control means 
are distributed within the shown configuration and are, 

30 in turn, controlled via feedback information from the 
processor in the block 21 (Fig. 5) . In response to the 
result of the monitoring and position data, such as the 
teat position information, which is calculated therein 
and which, among other things, is dependent on light 

35 conditions when recording the image information and the 
general interference level thereof, the control means 33 
are supplied with feedback information which can be used 
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to control said selection of functions and settings in 
the blocks 23-32. 

The configuration in Fig. 6 is arranged to operate 
with a constant pixel delay irrespective of the selected 
5 mode of operation. This is accomplished by a number of 
controllable D flip-flops (not shown), which delay the 
signals such that they always reach the outputs with the 
same consumption of time. 

Fig. 7 shows the routine in the device according to 
10 the invention in an imaginary case of use for 

establishing a characteristic colour field in the area of 
residence, for example preparing the teat position 
information. The following steps are included: 



15 bl . According to the circumstances in the area of 

residence, for example the milking stations, such as 
varying light conditions, the signal processing can 
be initiated with a background measurement, the 
result of which can be used for setting selectable 
20 functions and parameters. 

b2. The look-up memory 26 is programmed with a suitable 

look-up table. 
b3. Selection of function for the calculation unit 27 
and the discriminator 28. The following selections 
25 are possible: 

b31- Deriving of the pertainment values and subse- 
quent discrimination; "pixels of interest" are 
advanced in coordinate form; used in case of 
minor signal interference and sharp edges of 
30 the colour field. 

b32- Transparent mode with discrimination; 

the pertainment values are not processed in the 
calculation unit 27 but are level-discriminated 
in the discriminator 28 and advanced in 
35 coordinate form; used in case of minor signal 

interference and blurred edges of the colour 
field. 
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b33- Transparent mode and bitmap representation; 

the pertainment values are not processed in the 
calculation unit 27 and the discriminator 28; 
the values are advanced in bitmap form; used in 
5 case of moderate signal interference. 

b34- Filtration and bitmap representation; 

the pertainment values are processed by averag- 
ing in the calculation unit 27 and advanced in 
bitmap form; used in case of major signal 
10 interference. 

b4 . On the basis of the pixel information selected under 
step b3 and supplied to the memory 20, the processor 
21 tries to calculate occurrence of the colour 
field, for example the animal or teat position 
15 information, for instance, by establishing the 

position of the animals or the teat tip in the field 
by "calculation of centre of mass". 

Has it been possible to establish the colour field? 
If "yes" (Y) , proceed to step b5; if "no" (N) , the 

20 processor selects an alternative calculation algo- 

rithm or, alternatively, returns the information 
to step b3, thereby affecting the possibilities of 
selection in this step. 
b5. The established colour field and, optionally, its 

25 position, such as the calculated teat position 

information, is supplied to the operational block 22 
in Fig. 5. The procedure then returns to step b3 for 
processing the next field. 

30 The flow diagram in Fig. 8 roughly shows the routine 

when preparing the look-up tables for programming of the 
look-up memory 26 in Fig. 6. A look-up table is generated 
based on a number of images of the features of interest, 
such as of animals in the area of residence or of the 

35 animal teats. 
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The following steps are included: 
al. Reading of an image. 

a2. Marking of characteristic colour fields of the 
animals or portions thereof, such as the teats. 

a3. Decision whether the material is sufficient for cal- 
culating a pertainment function. 

If "yes" (Y) , proceed to the next step; if "no" (N) , 
return to step al for reading a further image. 
a4. Colour analysis of the images in respect of colour 
tones included and calculation of the corresponding 
pertainment values for preparing a look-up table of 
such pertainment values. The pertainment value for 
a certain colour tone can be calculated as, for in- 
stance, the relationship between the number of 
pixels within marked surfaces or colour fields 
containing the tone and the total number of pixels 
containing the tone. Finally, the colour tone volume 
defined by the generated look-up table can be given 
approximating border zones by supplementing with 
colour tones that are not to be found in the used 
image material. 

In this manner, the required number of look-up 
tables for enabling programming of the look-up memory in 
dependence on the situation of use are prepared. 

As is evident from that stated above, the described 
embodiment is directed to the use of two video cameras. 
However, it is also possible within the scope of the 
invention to use a single video camera for "roughly 
locating" an animal or a teat in the vertical and the 
horizontal direction by using the described image 
analysing technique, and then "finely locating" the 
object of interest also in depth by means of other 
equipment, for instance, a laser which generates on the 
object of interest a measure-indicating laser line, which 
can be detected with the aid of the same video camera. 
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The invention may of course also be used in circum- 
stances when no depth information is required and conse- 
quently no depth information data need to be generated 
for the above mentioned calculating steps. 
5 Fig. 9 shows an example of a general arrangement for 

recording the monitoring information according to the 
invention. A video camera 51 is arranged to scan an area 
of residence which is represented by the surface 52. 
• Within the area of residence there are five animals, four 

10 of which are represented by shaded circles 53, and one 
animal is represented by a filled circle 54. In this 
case, it is assumed that the animal 54 has been subjected 
to some kind of injury that has caused a characteristic 
change of colour, for instance a bleeding wound. . 

15 In a first case, the device according to the inven- 

tion can be used to establish that said animal actually 
is in the area by establishing a characteristic colour. 
In a second case, a rough determination of the position 
of the animal in the area of residence can be established 

20 as the point of intersection between an imaginary line 55 
from the camera that intersects the surface of the area 
of residence, based on the knowledge of the momentary 
direction of the camera 51, the position of the camera 
in relation to the area of residence, and a calculated 

25 position of the centre of mass for selected pixels repre- 
senting the animals in the manner as described above. The 
detailed course of calculation must be considered obvious 
to those skilled in the art and therefore is ignored in 
this context. 

30 In a further case, the device according to the in- 

vention can be used to establish whether and, optionally, 
where an animal with a bleeding wound is to be found in 
the area of residence. The look-up memory unit 26 is then 
programmed with a look-up table defining the colour tones 

35 that apply to bleeding wounds. Otherwise, the processing 
of the recorded image information proceeds in the manner 
described above. If an injured animal is established to 
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be within the area of residence, this information can be 
used to release an alarm signal. 

It will be appreciated by the expert that many 
variations of the embodiments described above are feas- 
ible within the scope of the invention. That described 
above should therefore not be considered restrictive to 
the scope of protection of the invention. 
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CLAIMS 

1 . A method in animal breeding for recording informa- 
tion on individual animals and/or collectives of animals 

5 which are within a defined area of residence, one or more 
video cameras (A, B; 17) being used to record video 
signals including image information from the animal or 
animals, and monitoring, position and/or positioning data 
, being extracted from the recorded image information, 
10 comprising the steps of: 

analysing the recorded image information in respect 
of this pixel related content of one or more colour tones 
being characteristic of one or more animals and/or one or 
more defined portions and/or states thereof; 
15 examining, in said analysis, the colour content of 

said pixels in respect of its pertinent to one or more 
predefined colour tone volumes; and 

calculating said data on the basis of pixels whose 
colour content falls within said one or more predefined 
20 colour tone volumes. 

2. A method according to claim 1, wherein said steps 
comprise: 

- producing the RGB signals included in the video signals 

2 5 in digital form for the pixels in an image frame or field 

of the image information; 

- analysing (26) the digital RGB signals for the pixels 
in respect of their content of said one or more colour 
tones which are characteristic of one or more animals 

30 and/or one or more defined portions and/or states there- 
of; 

- identifying each characteristic colour tone by applying 
a pertainment function (26) , the colour content of 

a pixel being tested for its pertainment to said one or 

3 5 more predefined colour tone volumes constituting one or 

more parts of a colour tone space which is formed by the 
colour intensities of the RGB signals; 
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- selecting pixels (26, 27, 28, 30), whose colour content 
falls within a colour tone volume; and 

- calculating said data (21) on the basis of the thus 
selected pixels. 

3. The method as claimed in claim 2, wherein each pixel, 
during testing for its pertainment to a colour tone 
volume, is given a corresponding pertainment value 
stating the position of the pixel in the colour tone 
volume and, thus, constituting a measure of the degree of 
relationship of the pixel. 



4. The method as claimed in claim 1,2 or 3, wherein: 

- the colour analysis is carried out in real time at 
15 video frequency; and 

- the colour analysis of a frame or field is carried out 
continuously while recording the frame or field. 



5. The method as claimed in any one of the preceding 
claims, wherein the colour analysis is carried out 
locally within a frame or field and line by line of the 
recorded image information. 

6. The method as claimed in any one of claims 1-4, 

25 wherein the colour analysis is carried out locally within 
a frame or field and comprises a number of successive 
lines in the image information. 

7. The method as claimed in any one of claims 2-7, 
30 wherein 

- the colour analysis of . the image information is carried 
out in a dedicated hardware (Fig. 6), in which each 
colour tone volume is defined by a look-up table of per- 
tainment values that are stored in a look-up memory unit 

35 (26); 

- the RGB intensity values of the pixels are supplied as 
address information to the look-up memory unit; and 
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- the pertainment value of a pixel is read from the 
memory when the RGB address information of the pixel 
corresponds to an address in the look-up table or, 
alternatively, the pertainment value zero or the 

5 equivalent is given to the pixel if the address is not 
comprised by the look-up table. 

8 . The method as claimed in any one of the preceding 
- claims, in which two spaced-apart video cameras are used 
10 to record the image information, wherein a further 

reduction of the data set is achieved by the following 
steps : 

- the currently monitored object is illuminated by two 
limited light beams of different colour, one light beam 

15 preferably containing light within the red colour spec- 
trum, while the other light beam contains light within 
the blue colour spectrum; 

- said light beams are directed to the object, such that 
the light beams are caused to intersect within an expect- 

20 ed area of residence of the object, the area of residence 
being predicted on the basis of historical data regarding 
the object; and 

- only pixels containing colour contributions from the 
two light beams are selected. 



9. The method as claimed in any one of claims 3-8, 
wherein the number of pertainment values obtained are 
reduced by the following steps: 

- derivation (27; ALU) of the pertainment values, the 
30 derivation preferably being carried out along the hori- 
zontal lines of the frame or field; and 

- discrimination (28; DISC) of derivative values below 
a predetermined threshold value. 
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10. The method as claimed in any one of claims 3-8, 
wherein the number of pertainment values obtained is 
reduced by the following steps: 

- successive averaging (27; ALU) of adjoining pertainment 
5 values; and 

- discrimination (2 8; DISC) of average values below a 
predetermined threshold value. 

11- The method as claimed in any one of claims 3-10, 
10 wherein: 

- the centre of mass is calculated (21; CPU) for the per- 
tainment values of the selected pixels, the calculation 
being based on the geometric positions of the pixels and 
the weight thereof, as represented by the associated per- 

15 tainment values; and 

- the position of a portion having a characteristic 
colour tone in the frame or field is selected to be equal 
to the position of the calculated centre of mass. 

20 . 12 • Tne method as claimed in any one of the preceding 

claims, for recording of monitoring information, wherein: 

- the area of residence is scanned by using one or more 
video cameras (Fig. 9) arranged for observing the area; 
and , 

25 - the position of a characteristic colour tone within the 
area of residence is calculated on the basis of the posi- 
tions of the selected pixels in a frame or field, the 
momentary direction and position of each video camera in 
relation to the area of residence. 

30 

13. The method as claimed in any one of the preceding 
claims, wherein: 

- said one or more colour tone volumes is/are charac- 
teristic of injuries or other abnormal changes of the 

35 animal or animals; and 

- an alarm signal is emitted on identification of such 
a colour tone. 
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14. The method as claimed in any one of the preceding 
claims, wherein: 

- the point of time for calculating position data is 
5 registered; and 

- an alarm signal is released when said data after a 
certain time assumes values that deviate from predeter- 
mined expected values. 

10 15. A method as claimed in any one of the preceding 

claims, the method being used at a milking station for 
extracting position information on the teats of an animal 
for guiding an animal-related device and said one or more 
video cameras being used for recording image information 

15 from the udder, wherein said steps comprise: 

- generating the digital RGB signals of the pixels in the 
recorded image information; 

- reducing the number of pixels by analysis (26) of the 
RGB signals with respect to the content of colour tones 

20 which are characteristic of the colour of the teats; 

- employing, in said analysis, a pertainment function 
(26), the colour content of the pixels being tested for 
its pertainment to a limited predefined colour tone 
volume of teat colour tones in an RGB intensity space and 

25 being given a pertainment value corresponding to the 
positions of the pixels in or outside the colour tone 
volume, 

- selecting (20; 26, 27, 28, 30) pixels having a colour 
tone that falls within the colour tone volume as probable 

30 teat pixels, and 

- using the pertainment values of the selected pixels for 
continued processing and real time calculation (21) of 
the positions of the teats. 
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16. The method as claimed in claim 15, wherein: 

- the pertainment values of the selected pixels are 
derived (27) for establishing colour tone transitions 
representing probable teat edges; and 

- derivative values below a predetermined threshold value 
are discriminated (28). 

17. The method as claimed in claim 15 or 16, wherein: 

- average values are formed (27) of the pertainment 
values for adjoining pixels; and 

- average values below a predetermined threshold value 
are discriminated (28). 

18. The method as claimed in claim 15, 16 or 17, 
wherein: 

- pixels spaced apart by a given number of pixels are 
selected (21) along each line in the frame or field 
represented by the pertainment values; 

- the amplitudes of thus selected pairs of values are 
compared (21) for establishing a probable intermediate 
teat edge transition; 

- pairs of values for which such a transition cannot be 
established are discriminated; and 

- the procedure -is repeated until a limited number of 
values are obtained which represent probable teat image 
or teat edge pixels. 

19. The method as claimed in claim 15, 16, 17 or 18, 
wherein the analysis is limited to pixels within a 
certain area round a teat position that has been 
established during a preceding frame or field. 

20. The method as claimed in any one of claims 15-19, 
wherein : 

- the teats are scanned from below and upwards by means 
of said camera (A; B; 17); and 
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- a frame or field of the image information is analysed 



continuously while the camera is scanning the teats. 

21. The method as claimed in any one of claims 15-20, 

5 wherein the image information is analysed locally and in 
line by line within a frame or field. 

22. The method as claimed in any one of claims 15-20 r 
- wherein the image information is analysed locally and 

10 comprises a number of successive lines in a frame or 
field. 

23. The method as claimed in any one of claims 15-22, 
wherein the pertainment function is implemented by a 

15 programmable look-up unit (26) containing a look-up table 
which defines said colour tone volume and contains a set 
of pertainment values. 

24. The method as claimed in claim 23, the milking sta- 
20 tion containing means for automatically reading out the 

identity of the animal, wherein: 

- the identity of the animal is read out by said means; 
and 

- the look-up unit (26) is programmed in real time with a 
25 look-up table of pertainment values, which is related to 

the read-out identity. 

25. The method as claimed in claim 24, wherein a unique 
look-up table is prepared in advance for each animal 

30 individual and stored in a database, the look-up table 
being collected from the database using the read-out 
identity information. 

26. The method as claimed in claim 24, wherein: 

35 - a look-up table of a group of animal individuals having 
similar teat colours is prepared in advance and stored in 
a database; and 
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- the look-up table is collected from the database using 
the read-out identity information of an animal belonging 
to the group . 

5 27. The method as claimed in any one of claims 23-26, 
wherein the look-up table is prepared by the following 
steps (Fig. 8) : 

- recording one or more colour images of one or more 
sets of teats of similar colour, 

10 - analysing the image or images in respect of colour 
tones included, 

- calculation of a pertainment value for each included 
colour tone as a measure of the uniqueness of the colour 
tone. for a teat image, the thus calculated pertainment 

15 values being used to form the look-up table. 

28. The method as claimed in any one of claims 15-27, 
wherein: 

- the teats are illuminated by two light beams of diffe- 
20 ( rent colour at a given relative angle (Fig. 4), one light 

beam preferably being within the red colour spectrum and 
the other light beam preferably being within the blue 
colour spectrum; 

- the light beams are directed so as to intersect within 
25 an expected zone of residence for the teats, said zone of 

residence being predicted based on historical data; 

- the pixels are tested for their content of the respec- 
tive colour of the light beams; and 

- pixels are selected which contain colour contributions 
30 from both light beams. 

29. The method as claimed in any one of claims 15-28, 
wherein the position of a teat tip (5a) is calculated 
based on selected pixels while using an algorithm for a 

35 centre of mass calculation, the calculation being based 
on the relative geometric positions of the pixels in a 
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frame or field and the weight of each pixel as represen- 
ted by its pertainment value. 

30. The method as claimed in any one of claims 15-29, 
5 wherein the position of a teat tip (5a) is calculated on 
the basis of selected pixels while using an algorithm 
which is based on prior art shape factors of the type 
teat length/width. 

10 31. The method as claimed in any one of claims 15-30, 

wherein the position of a teat tip (5a) is calculated on 
the basis of selected pixels while using an algorithm 
which is based on the position of the udder and 
historical data regarding the teat positions. 

15 

32. The method as claimed in any one of claims 15-31, 
wherein: 

- the image information is recorded by two video cameras 
(A, B; 17) which are arranged in a spaced-apart relation- 
20 ship preferably in the same horizontal plane and on the 
same side of the animal, and adapted to watch the udder 
obliquely from the front between the forelegs and hind 
legs of the animal; 



25 analysed in parallel and in corresponding manners; 

- for each camera, the X and Y coordinates for a teat tip 
are calculated on the basis of selected pixels; and 

- the 2 coordinate of the teat tip is calculated on the 
basis of the X and Y coordinates from the two cameras, 

30 the relative distance between the cameras and the direc- 
tion the cameras. 



the image information from the respective cameras is 
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33. A device in animal breeding for calculating monito- 
ring, position and/or positioning data for individual 
animals and/or collectives of animals which are within a 
defined area of residence, comprising one or more video 
cameras (A, B; 17) for recording video signals including 
image information from the animal or animals and/or said 
area of residence, said device comprising: 

means (19) for analysing the recorded image 
information in respect of its pixel related content of 
one or more colour tones being characteristic of one or 
more animals and/or one or more defined portions and/or 
states thereof; 

means (19) for examining the colour content of said 
pixels in respect of its pertinent to one or more pre- 
15 defined colour tone volumes; and 

means (21) for calculating said data on the basis of 
pixels whose colour content falls within said one or more 
predefined colour tone volumes. 
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34. A device as claimed in claim 33, comprising: 

- signal converting means (18) for producing in digital 
form the RGB signals for the pixels in the pictures or 
partial pictures of the image information; 

- a data reducing unit (19) for reducing the amount of 
information in each frame or field by colour analysis of 
the pixels, comprising a pertainment function unit to 
which the RGB signals are supplied for testing the rela- 
tionship of each pixel with a defined, animal-character- 
istic colour tone volume in an RGB colour intensity space 
and providing for every pixel a pertainment value, and 
means for selecting pixels having a colour tone falling 
within the colour tone volume; 

- memory means (20) for storing the pertainment values 
of the selected pixels and the positions thereof in the 

35 frame or field; 

- calculating means (21) for calculating said data on the 
basis of the stored information in said memory means, and 
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- means (21, 33) for controlling the mode of operation of 
the device. 

35. The device as claimed in claim 34, wherein: 
5 - the pertainment function unit is accomplished by a 
look-up memory (26) ; 

- the look-up memory is programmable with a look-up table 
containing addressable pertainment values which define 

_ said colour tone volume, the size of the pertainment 
10 value stating the degree by which a colour tone is char- 
acteristic of a colour of the animal or animals; and 

- the look-up memory is provided with address inputs for 
supplying thereto the RGB signals as address information. 

15 36. The device as claimed in claim 34 or 35, the image 
information containing colour fields with defined edges, 
wherein the data reducing unit (19) comprises differen- 
tiating means (27) for deriving the pertainment values of 
the selected pixels, said selecting means comprising a 

20 discriminator (28) for suppressing derivatives below a 
threshold value and, thus, selecting pixels related to 
said edges. 

37. The device as claimed in any one of the preceding 
25 claims 34-36, wherein: 

- said memory means (20) comprises a direct access memory 
having double input and output ports; and 

- said calculating means (21) is adapted to carry out 
output data calculations for a frame or field in parallel 

30 with reading of the selected pixels in the memory means. 

38. The device as claimed in any one of the preceding 
claims 34-37, wherein: 

- the data reducing unit (19) is adapted to operate with 
35 a constant pixel delay, controllable delay means being 

adapted to compensate for varying consumption of time 
for selectable processing routines; and 



WO 97/15901 W 40 _ PCT/SE96/01368 



10 



- said calculating means (21) is adapted to measure the 
calculating time used, 

whereby the device obtains a deterministic pixel delay, 
which permits correction of the calculated position data 
5 in respect of the movements of the animal /animals . 

39. The device as claimed in any one of claims 33-38, 
wherein said calculating means (21) is programmed to 
carry out a centre of mass calculation for selected 
pixels based on the geometric positions of the elements 
and the weight thereof as represented by the associated 
pertainment values, the coordinates of a defined colour 
field in the frame or field being set equal to the 
coordinates of the centre of mass and constituting the 

15 basis for calculating said position data. 

40. A device as claimed in any one of claims 33-39, 
provided at a milking station comprising animal-related 
means (6, 11, 12, 13), which are guidable towards the 

20 , teats (5) of an animal (1) in dependence on position 

information on the teats, said one or more video cameras 
(A; B; 17) being adapted for recording image information 
from the teats and/or udder of the animal, wherein said 
device comprises the following function units: 

25 - signal converting means (18) for preparing the digital 
RGB signals of the pixels in the recorded image informa- 
tion; 

- data reducing means (19) for reducing the number of 
pixels by analysis of the RGB signals with respect to the 
content of teat-characteristic colour tones, comprising a 
pertainment function unit (26) to which the RGB signals 
are supplied and which is adapted to test the colour con- 
tent of the pixels for their pertainment to a limited 
colour tone volume, defined in advance, of teat colour 
35 tones in an RGB intensity space, and to give the pixels 
pertainment values in response to their positions in or 
outside the colour tone volume, and means (27, 28) for 
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selecting pixels having a colour tone falling inside the 
colour tone volume; 

- memory means (20) for storing the pertainment values 
and positions of selected pixels in a field or frame of 

5 the image information; 

- calculating means (21) for calculating the position 
information based on the stored information in said memo- 
ry means; and 

_ - means (21, 33) for controlling the mode of operation of 
10 the device. 

41. The device as claimed in claim 40, wherein: 

- the pertainment function unit (26) is implemented by 
a look-up memory which is programmable with selectable 

15 look-up tables of pertainment values, each table defining 
a colour tone volume, and 

- the look-up memory is provided with address inputs for 
receiving the RGB signals as address information and 
reading the corresponding pertainment values. 



42. The device as claimed in claim 41, having access to 
a database for storing look-up tables prepared in 
advance, each table being unique for an animal individual 
or, alternatively, a group of animal individuals having 

25 similar teat colours, and the milking station being 

provided with means for reading out the identity of the 
animal, wherein the look-up memory is adapted, on the 
occasion of milking, to be programmed with the relevant 
look-up table, which is collected from said database in 

30 dependence on the read-out identity information. 

43. The device as claimed in any one of claims 40-42, 
wherein: 

- said data reducing means (19) comprises differentiating 
35 means (27) adapted to derive the pertainment values of 
selected pixels; and 
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- said selecting means comprises a discriminator (28) 
adapted to suppress derivatives below a threshold value, 
whereby pixels related to teat edges are selected. 

5 44. The device as claimed in any one of claims 40-43, 
wherein: 

- said memory means (20) comprises a direct access memory 
having double input and output ports; and 

- said calculating means (21) is adapted to carry out the 
10 calculation of the position information in parallel with 

the current reading of pertainment value information for 
subsequent pixels. 

45. The device as claimed in any one of claims 40-44, 
wherein: 

- said data reducing means (19) are implemented by a 
dedicated hardware, which is adapted to operate with a 
constant pixel delay, and which comprises controllable 
delay means for compensating variable consumption of time 
of different selectable processing procedures; and 

- said calculating means (21) is adapted to measure the 
used calculating time, 

whereby the processing of each field takes place with a 
deterministic pixel delay, which enables correction of 
the calculated position information in respect of the 
movements of the teats. 

46. The device as claimed in any one of claims 40-45, 
wherein: 

- two light sources (C, D) are arranged to direct light 
beams (C 1 , D 1 ) of different colours, directed such that 
the light beams intersect within a predicted teat area; 
and 

- the colour tone volume is defined to contain only 
colour tones having contributions from both light beams, 
whereby the foreground and background pixels having teat- 
characteristic colour tones can be. discriminated. 



15 



20 



25 



WO 97/15901 ^ 43 — PCT/SE96/01368 



47. The device as claimed in any one of claims 40-46, 
comprising a controllable robot arm (7) for guiding said 
animal-related means, wherein at least one video camera 

5 (A; B) is arranged on the robot arm. 

48. The milking station as claimed in any one of claims 
40-47, wherein: 

. - two video cameras (A, B) which are arranged in a 
10 spaced-apart relationship, preferably in the same hori- 
zontal plane and on the same side of the animal, and are 
adapted to watch the udder obliquely from the front 
between the forelegs and hind legs of the animal; and 
- said device for processing the video information of the 
15 cameras being doubled, 

whereby the image information from the respective cameras 
is analysed in parallel and in corresponding manners. 

49. Use of the method and/or the device according to any 
20 one of the preceding claims in equipment (22) related to 

animals in connection with supervising, foddering and 
milking of cows. 

50. Use of the method and/or the device according to any 
25 one of the preceding claims for automatically applying 

teat cups at a milking and/or foddering station. 
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